Consideraciones bioéticas en el manejo de la esclerosis lateral amiotrófica: Una aproximación al quehacer fonoaudiológico

Introducción: La esclerosis lateral amiotrófica (ELA) es la forma más común de enfermedad degenerativa de motoneurona en la edad adulta y es considerada una enfermedad terminal. Por lo mismo, el accionar del fonoaudiólogo debe considerar el respeto a los principios bioéticos básicos para garantizar una asistencia adecuada. Objetivo: Conocer aquellas consideraciones bioéticas relacionadas al manejo y estudio de personas con ELA para luego brindar una aproximación hacia el quehacer fonoaudiológico. Método: Se efectuó una búsqueda bibliográfica en las bases de datos PubMed, Scopus y SciELO. Se filtraron artículos publicados desde 2000 hasta junio de 2023 y fueron seleccionados aquellos que abordaban algún componente bioético en población con ELA. Resultados: Aspectos relacionados al uso del consentimiento informado y a la toma de decisiones compartidas destacaron como elementos esenciales para apoyar la autonomía de las personas. Conclusión: Una correcta comunicación y una toma de decisiones compartida son claves para respetar la autonomía de las personas. A su vez, la estandarización de procedimientos mediante la investigación clínica permitirá aportar al cumplimiento de los principios bioéticos de beneficencia y no maleficencia, indispensables para la práctica profesional.

Introduction: Amyotrophic lateral sclerosis (ALS) is the most common form of degenerative motor neuron disease in adulthood and is considered a terminal disease. For this reason, the actions of the speech therapist must consider respect for basic bioethical principles to guarantee adequate assistance. Objective: To know those bioethical considerations related to the management and study of people with ALS to then provide an approach to speech therapy. Methodology: A bibliographic search was carried out in the PubMed, Scopus, and SciELO databases. Articles published from 2000 to June 2023 were filtered and those that addressed a bioethical component in the population with ALS were selected. Results: Aspects related to the use of informed consent and shared decision-making stood out as essential elements to support people's autonomy. Conclusion: Proper communication and shared decision-making are key to respecting people's autonomy. In turn, the standardization of procedures through clinical research will contribute to compliance with the bioethical principles of beneficence and non-maleficence, essential for professional practice.

Outcomes of a Community of Practice on Quebec Speech Language Pathologists' Voice Assessment Practices and Professional Identity

In a context where different protocols for recommended practices in clinical voice assessment exist, while there are gaps in the literature regarding the evidence base supporting assessment procedures and measures, clinicians from regions where a strong community holding expertise in clinical and scientific voice practices lack can struggle to confidently develop their voice assessment practices. In an effort to improve voice assessment practices and strengthen professional identity among speech-language pathologists in Quebec, Canada, a community of practice (CoP) was established, with the aim of promoting knowledge sharing, implementing change in clinical practice, and improving professional identity. Thirty-nine participants took part in the CoP activities conducted over a four-month period, including virtual meetings and in-person workshops. Participants had a high rate of attendance (> 74% participation rate in virtual meetings), and were highly satisfied with their participation and intended to remain involved after the project's end. Statistically significant changes in voice assessment practices were observed post-CoP, regarding probability of performing assessments (p < .001), and perceived importance of assessment for evaluative purposes (p <.001), as well as improvements in assessment specific confidence, specifically for procedure of auditory-perceptual assessment (p < .001) and purpose of aerodynamic assessment (p = .05). Moreover, there was an increase in professional identity post-CoP (p < .001) and participants felt they made significant learnings. The present study highlighted the need to involve SLPs in future research to identify assessments that are relevant to the specific evaluative objectives of SLPs working with voice, and suggests CoPs are an efficient tool for that purpose.

En un contexto en el que existen diferentes protocolos para las prácticas recomendadas en la evaluación vocal clínica, y en el que se presentan vacíos en la literatura respecto a la base de evidencia que respalda los procedimientos y medidas de evaluación, los profesionales de regiones donde no hay una comunidad sólida con experiencia en prácticas vocales clínicas y científicas pueden enfrentar dificultades para desarrollar con confianza sus prácticas de evaluación vocal. Con el propósito de mejorar las prácticas de evaluación vocal y fortalecer la identidad profesional entre los logopedas de Quebec, Canadá, se estableció una comunidad de práctica (CdP). Esta tenía como objetivo fomentar el intercambio de conocimientos, implementar cambios en la práctica clínica y mejorar la identidad profesional. Un total de treinta y nueve participantes se involucraron en las actividades de la CdP, llevadas a cabo durante un período de cuatro meses, que incluyeron reuniones virtuales y talleres presenciales. Los participantes tuvieron una alta tasa de asistencia (> 74% de participación en las reuniones virtuales) y expresaron un alto grado de satisfacción con su participación, manifestando su intención de continuar involucrados después de la finalización del proyecto. Se observaron cambios estadísticamente significativos en las prácticas de evaluación vocal posterior a la CdP, en lo que respecta a la probabilidad de llevar a cabo evaluaciones (p < .001) y la percepción de la importancia de la evaluación con fines evaluativos (p < .001), así como mejoras en la confianza específica en la evaluación, particularmente en el procedimiento de evaluación auditivo-perceptual (p < .001) y el propósito de la evaluación aerodinámica (p = .05). Además, se registró un aumento en la identidad profesional posterior a la CdP (p < .001) y los participantes sintieron que obtuvieron aprendizajes significativos. El presente estudio destacó la necesidad de involucrar a los logopedas en investigaciones futuras, para identificar evaluaciones pertinentes a los objetivos evaluativos específicos de los logopedas que trabajan con la voz, y sugiere que las CdP son una herramienta eficiente con ese propósito.

KEGG orthology prediction of bacterial proteins using natural language processing.

BACKGROUND: The advent of high-throughput technologies has led to an exponential increase in uncharacterized bacterial protein sequences, surpassing the capacity of manual curation. A large number of bacterial protein sequences remain unannotated by Kyoto Encyclopedia of Genes and Genomes (KEGG) orthology, making it necessary to use auto annotation tools. These tools are now indispensable in the biological research landscape, bridging the gap between the vastness of unannotated sequences and meaningful biological insights. RESULTS: In this work, we propose a novel pipeline for KEGG orthology annotation of bacterial protein sequences that uses natural language processing and deep learning. To assess the effectiveness of our pipeline, we conducted evaluations using the genomes of two randomly selected species from the KEGG database. In our evaluation, we obtain competitive results on precision, recall, and F1 score, with values of 0.948, 0.947, and 0.947, respectively. CONCLUSIONS: Our experimental results suggest that our pipeline demonstrates performance comparable to traditional methods and excels in identifying distant relatives with low sequence identity. This demonstrates the potential of our pipeline to significantly improve the accuracy and comprehensiveness of KEGG orthology annotation, thereby advancing our understanding of functional relationships within biological systems.

Intersectional race-gender stereotypes in natural language.

How are Asian and Black men and women stereotyped? Research from the gendered race and stereotype content perspectives has produced mixed empirical findings. Using BERT models pre-trained on English language books, news articles, Wikipedia, Reddit and Twitter, with a new method for measuring propositions in natural language (the Fill-Mask Association Test, FMAT), we explored the gender (masculinity-femininity), physical strength, warmth and competence contents of stereotypes about Asian and Black men and women. We find that Asian men (but not women) are stereotyped as less masculine and less moral/trustworthy than Black men. Compared to Black men and Black women, respectively, both Asian men and Asian women are stereotyped as less muscular/athletic and less assertive/dominant, but more sociable/friendly and more capable/intelligent. These findings suggest that Asian and Black stereotypes in natural language have multifaceted contents and gender nuances, requiring a balanced view integrating the gender schema theory and the stereotype content model. Exploring their semantic representations as propositions in large language models, this research reveals how intersectional race-gender stereotypes are naturally expressed in real life.

Exploiting protein language models for the precise classification of ion channels and ion transporters.

This study introduces TooT-PLM-ionCT, a comprehensive framework that consolidates three distinct systems, each meticulously tailored for one of the following tasks: distinguishing ion channels (ICs) from membrane proteins (MPs), segregating ion transporters (ITs) from MPs, and differentiating ICs from ITs. Drawing upon the strengths of six Protein Language Models (PLMs)-ProtBERT, ProtBERT-BFD, ESM-1b, ESM-2 (650M parameters), and ESM-2 (15B parameters), TooT-PLM-ionCT employs a combination of traditional classifiers and deep learning models for nuanced protein classification. Originally validated on an existing dataset by previous researchers, our systems demonstrated superior performance in identifying ITs from MPs and distinguishing ICs from ITs, with the IC-MP discrimination achieving state-of-the-art results. In light of recommendations for additional validation, we introduced a new dataset, significantly enhancing the robustness and generalization of our models across bioinformatics challenges. This new evaluation underscored the effectiveness of TooT-PLM-ionCT in adapting to novel data while maintaining high classification accuracy. Furthermore, this study explores critical factors affecting classification accuracy, such as dataset balancing, the impact of using frozen versus fine-tuned PLM representations, and the variance between half and full precision in floating-point computations. To facilitate broader application and accessibility, a web server (https://tootsuite.encs.concordia.ca/service/TooT-PLM-ionCT) has been developed, allowing users to evaluate unknown protein sequences through our specialized systems for IC-MP, IT-MP, and IC-IT classification tasks.

Mining Clinical Notes for Physical Rehabilitation Exercise Information: Natural Language Processing Algorithm Development and Validation Study.

BACKGROUND: The rehabilitation of a patient who had a stroke requires precise, personalized treatment plans. Natural language processing (NLP) offers the potential to extract valuable exercise information from clinical notes, aiding in the development of more effective rehabilitation strategies. OBJECTIVE: This study aims to develop and evaluate a variety of NLP algorithms to extract and categorize physical rehabilitation exercise information from the clinical notes of patients who had a stroke treated at the University of Pittsburgh Medical Center. METHODS: A cohort of 13,605 patients diagnosed with stroke was identified, and their clinical notes containing rehabilitation therapy notes were retrieved. A comprehensive clinical ontology was created to represent various aspects of physical rehabilitation exercises. State-of-the-art NLP algorithms were then developed and compared, including rule-based, machine learning-based algorithms (support vector machine, logistic regression, gradient boosting, and AdaBoost) and large language model (LLM)-based algorithms (ChatGPT [OpenAI]). The study focused on key performance metrics, particularly F1-scores, to evaluate algorithm effectiveness. RESULTS: The analysis was conducted on a data set comprising 23,724 notes with detailed demographic and clinical characteristics. The rule-based NLP algorithm demonstrated superior performance in most areas, particularly in detecting the "Right Side" location with an F1-score of 0.975, outperforming gradient boosting by 0.063. Gradient boosting excelled in "Lower Extremity" location detection (F1-score: 0.978), surpassing rule-based NLP by 0.023. It also showed notable performance in the "Passive Range of Motion" detection with an F1-score of 0.970, a 0.032 improvement over rule-based NLP. The rule-based algorithm efficiently handled "Duration," "Sets," and "Reps" with F1-scores up to 0.65. LLM-based NLP, particularly ChatGPT with few-shot prompts, achieved high recall but generally lower precision and F1-scores. However, it notably excelled in "Backward Plane" motion detection, achieving an F1-score of 0.846, surpassing the rule-based algorithm's 0.720. CONCLUSIONS: The study successfully developed and evaluated multiple NLP algorithms, revealing the strengths and weaknesses of each in extracting physical rehabilitation exercise information from clinical notes. The detailed ontology and the robust performance of the rule-based and gradient boosting algorithms demonstrate significant potential for enhancing precision rehabilitation. These findings contribute to the ongoing efforts to integrate advanced NLP techniques into health care, moving toward predictive models that can recommend personalized rehabilitation treatments for optimal patient outcomes.

Leveraging Large Language Models for Improved Patient Access and Self-Management: Assessor-Blinded Comparison Between Expert- and AI-Generated Content.

BACKGROUND: While large language models (LLMs) such as ChatGPT and Google Bard have shown significant promise in various fields, their broader impact on enhancing patient health care access and quality, particularly in specialized domains such as oral health, requires comprehensive evaluation. OBJECTIVE: This study aims to assess the effectiveness of Google Bard, ChatGPT-3.5, and ChatGPT-4 in offering recommendations for common oral health issues, benchmarked against responses from human dental experts. METHODS: This comparative analysis used 40 questions derived from patient surveys on prevalent oral diseases, which were executed in a simulated clinical environment. Responses, obtained from both human experts and LLMs, were subject to a blinded evaluation process by experienced dentists and lay users, focusing on readability, appropriateness, harmlessness, comprehensiveness, intent capture, and helpfulness. Additionally, the stability of artificial intelligence responses was also assessed by submitting each question 3 times under consistent conditions. RESULTS: Google Bard excelled in readability but lagged in appropriateness when compared to human experts (mean 8.51, SD 0.37 vs mean 9.60, SD 0.33; P=.03). ChatGPT-3.5 and ChatGPT-4, however, performed comparably with human experts in terms of appropriateness (mean 8.96, SD 0.35 and mean 9.34, SD 0.47, respectively), with ChatGPT-4 demonstrating the highest stability and reliability. Furthermore, all 3 LLMs received superior harmlessness scores comparable to human experts, with lay users finding minimal differences in helpfulness and intent capture between the artificial intelligence models and human responses. CONCLUSIONS: LLMs, particularly ChatGPT-4, show potential in oral health care, providing patient-centric information for enhancing patient education and clinical care. The observed performance variations underscore the need for ongoing refinement and ethical considerations in health care settings. Future research focuses on developing strategies for the safe integration of LLMs in health care settings.

Comparison of minimally invasive to standard temporal lobectomy approaches to epilepsy surgery: Seizure relief and visual confrontation naming outcomes.

The purpose of this study was to systematically examine three different surgical approaches in treating left medial temporal lobe epilepsy (mTLE) (viz., subtemporal selective amygdalohippocampectomy [subSAH], stereotactic laser amygdalohippocampotomy [SLAH], and anterior temporal lobectomy [ATL]), to determine which procedures are most favorable in terms of visual confrontation naming and seizure relief outcome. This was a retrospective study of 33 adults with intractable mTLE who underwent left temporal lobe surgery at three different epilepsy surgery centers who also underwent pre-, and at least 6-month post-surgical neuropsychological testing. Measures included the Boston Naming Test (BNT) and the Engel Epilepsy Surgery Outcome Scale. Fisher's exact tests revealed a statistically significant decline in naming in ATLs compared to SLAHs, but no other significant group differences. 82% of ATL and 36% of subSAH patients showed a significant naming decline whereas no SLAH patient (0%) had a significant naming decline. Significant postoperative naming improvement was seen in 36% of SLAH patients in contrast to 9% improvement in subSAH patients and 0% improvement in ATLs. Finally, there were no statistically significant differences between surgical approaches with regard to seizure freedom outcome, although there was a trend towards better seizure relief outcome among the ATL patients. Results support a possible benefit of SLAH in preserving visual confrontation naming after left TLE surgery. While result interpretation is limited by the small sample size, findings suggest outcome is likely to differ by surgical approach, and that further research on cognitive and seizure freedom outcomes is needed to inform patients and providers of potential risks and benefits with each.

Continuous white noise exposure during sleep and childhood development: A scoping review.

BACKGROUND: White noise machines are widely used as a sleep aid for young children and may lead to poor hearing, speech, and learning outcomes if used incorrectly. OBJECTIVE: Characterize the potential impact of chronic white noise exposure on early childhood development. METHODS: Embase, Ovid MEDLINE, the Cochrane Central Register of Controlled Trials, Scopus, and Web of Science were searched from inception through June 2022 for publications addressing the effects of chronic noise exposure during sleep on early development in animals and children. PRISMA-ScR guidelines were followed. Among 644 retrieved publications, 20 met inclusion criteria after review by multiple authors. Seven studies evaluated animal models and 13 studies examined pediatric subjects, including 83 animal and 9428 human subjects. RESULTS: White noise machines can exceed 91 dB on maximum volume, which exceeds the National Institute for Occupational Safety and Health noise exposure guidelines for a 2-h work shift in adults. Evidence suggests deleterious effects of continuous moderate-intensity white noise exposure on early development in animal models. Human subject data generally corroborates these models; however, studies also suggest low-intensity noise exposure may be beneficial during sleep. CONCLUSIONS: Existing data support the limitation of maximal sound intensity and duration on commercially available white noise devices. Further research into the optimal intensity and duration of white noise exposure in children is needed.

A systematic review of factors that impact reading comprehension in children with developmental language disorders.

Children with developmental language disorder (DLD) have a high rate of co-occurring reading difficulties. The current study aims to (i) examine which factors within the Active View of Reading (AVR; Duke & Cartwright, 2021) apply to individuals with DLD and (ii) investigate other possible factors that relate to reading comprehension ability in individuals with DLD, outside the components in the AVR. Electronic database search and journal hand-search yielded 5058 studies published before March 2022 related to reading comprehension in children with DLD. 4802 articles were excluded during abstract screening, yielding 256 studies eligible for full-text review. Following full-text review, 44 studies were included and further coded for demographics, language of assessment, description of reported disabilities, behavioral assessment, and reading comprehension assessment. While the results aligned with the AVR model, three additional factors were identified as significantly relating to reading comprehension abilities in children with DLD: expressive language (oral and written), question types of reading assessment, and language disorder history. Specifically, expressive language was positively associated with reading comprehension ability, while resolved DLD showed higher reading comprehension abilities than persistent DLD. Furthermore, children with DLD may face additional difficulties in comprehending inference-based questions. This study provides factors for researchers, educators, and clinical professionals to consider when evaluating the reading comprehension of individuals with DLD. Future research should further explore the relative importance of factors of the AVR to reading comprehension outcomes throughout development.

A natural language model to automate scoring of autobiographical memories.

Biases in the retrieval of personal, autobiographical memories are a core feature of multiple mental health disorders, and are associated with poor clinical prognosis. However, current assessments of memory bias are either reliant on human scoring, restricting their administration in clinical settings, or when computerized, are only able to identify one memory type. Here, we developed a natural language model able to classify text-based memories as one of five different autobiographical memory types (specific, categoric, extended, semantic associate, omission), allowing easy assessment of a wider range of memory biases, including reduced memory specificity and impaired memory flexibility. Our model was trained on 17,632 text-based, human-scored memories obtained from individuals with and without experience of memory bias and mental health challenges, which was then tested on a dataset of 5880 memories. We used 20-fold cross-validation setup, and the model was fine-tuned over BERT. Relative to benchmarking and an existing support vector model, our model achieved high accuracy (95.7%) and precision (91.0%). We provide an open-source version of the model which is able to be used without further coding, by those with no coding experience, to facilitate the assessment of autobiographical memory bias in clinical settings, and aid implementation of memory-based interventions within treatment services.

Improvements in viral gene annotation using large language models and soft alignments.

BACKGROUND: The annotation of protein sequences in public databases has long posed a challenge in molecular biology. This issue is particularly acute for viral proteins, which demonstrate limited homology to known proteins when using alignment, k-mer, or profile-based homology search approaches. A novel methodology employing Large Language Models (LLMs) addresses this methodological challenge by annotating protein sequences based on embeddings. RESULTS: Central to our contribution is the soft alignment algorithm, drawing from traditional protein alignment but leveraging embedding similarity at the amino acid level to bypass the need for conventional scoring matrices. This method not only surpasses pooled embedding-based models in efficiency but also in interpretability, enabling users to easily trace homologous amino acids and delve deeper into the alignments. Far from being a black box, our approach provides transparent, BLAST-like alignment visualizations, combining traditional biological research with AI advancements to elevate protein annotation through embedding-based analysis while ensuring interpretability. Tests using the Virus Orthologous Groups and ViralZone protein databases indicated that the novel soft alignment approach recognized and annotated sequences that both blastp and pooling-based methods, which are commonly used for sequence annotation, failed to detect. CONCLUSION: The embeddings approach shows the great potential of LLMs for enhancing protein sequence annotation, especially in viral genomics. These findings present a promising avenue for more efficient and accurate protein function inference in molecular biology.

Brain-computer interfaces and human factors: the role of language and cultural differences-Still a missing gap?

Brain-computer interfaces (BCIs) aim at the non-invasive investigation of brain activity for supporting communication and interaction of the users with their environment by means of brain-machine assisted technologies. Despite technological progress and promising research aimed at understanding the influence of human factors on BCI effectiveness, some topics still remain unexplored. The aim of this article is to discuss why it is important to consider the language of the user, its embodied grounding in perception, action and emotions, and its interaction with cultural differences in information processing in future BCI research. Based on evidence from recent studies, it is proposed that detection of language abilities and language training are two main topics of enquiry of future BCI studies to extend communication among vulnerable and healthy BCI users from bench to bedside and real world applications. In addition, cultural differences shape perception, actions, cognition, language and emotions subjectively, behaviorally as well as neuronally. Therefore, BCI applications should consider cultural differences in information processing to develop culture- and language-sensitive BCI applications for different user groups and BCIs, and investigate the linguistic and cultural contexts in which the BCI will be used.

Natural language processing (NLP) to facilitate abstract review in medical research: the application of BioBERT to exploring the 20-year use of NLP in medical research.

BACKGROUND: Abstract review is a time and labor-consuming step in the systematic and scoping literature review in medicine. Text mining methods, typically natural language processing (NLP), may efficiently replace manual abstract screening. This study applies NLP to a deliberately selected literature review problem, the trend of using NLP in medical research, to demonstrate the performance of this automated abstract review model. METHODS: Scanning PubMed, Embase, PsycINFO, and CINAHL databases, we identified 22,294 with a final selection of 12,817 English abstracts published between 2000 and 2021. We invented a manual classification of medical fields, three variables, i.e., the context of use (COU), text source (TS), and primary research field (PRF). A training dataset was developed after reviewing 485 abstracts. We used a language model called Bidirectional Encoder Representations from Transformers to classify the abstracts. To evaluate the performance of the trained models, we report a micro f1-score and accuracy. RESULTS: The trained models' micro f1-score for classifying abstracts, into three variables were 77.35% for COU, 76.24% for TS, and 85.64% for PRF. The average annual growth rate (AAGR) of the publications was 20.99% between 2000 and 2020 (72.01 articles (95% CI: 56.80-78.30) yearly increase), with 81.76% of the abstracts published between 2010 and 2020. Studies on neoplasms constituted 27.66% of the entire corpus with an AAGR of 42.41%, followed by studies on mental conditions (AAGR = 39.28%). While electronic health or medical records comprised the highest proportion of text sources (57.12%), omics databases had the highest growth among all text sources with an AAGR of 65.08%. The most common NLP application was clinical decision support (25.45%). CONCLUSIONS: BioBERT showed an acceptable performance in the abstract review. If future research shows the high performance of this language model, it can reliably replace manual abstract reviews.

Mobile device screen time is associated with poorer language development among toddlers: results from a large-scale survey.

BACKGROUND: Despite young children's widespread use of mobile devices, little research exists on this use and its association with children's language development. The aim of this study was to examine the associations between mobile device screen time and language comprehension and expressive language skills. An additional aim was to examine whether three factors related to the domestic learning environment modify the associations. METHODS: The study uses data from the Danish large-scale survey TRACES among two- and three-year-old children (n = 31,125). Mobile device screen time was measured as time spent on mobile devices on a normal day. Measurement of language comprehension and expressive language skills was based on subscales from the Five to Fifteen Toddlers questionnaire. Multivariable linear regression was used to examine the association between child mobile device screen time and language development and logistic regression to examine the risk of experiencing significant language difficulties. Joint exposure analyses were used to examine the association between child mobile device screen time and language development difficulties in combination with three other factors related to the domestic learning environment: parental education, reading to the child and child TV/PC screen time. RESULTS: High mobile device screen time of one hour or more per day was significantly associated with poorer language development scores and higher odds for both language comprehension difficulties (1-2 h: AOR = 1.30; ≥ 2 h: AOR = 1.42) and expressive language skills difficulties (1-2 h: AOR = 1.19; ≥ 2 h: AOR = 1.46). The results suggest that reading frequently to the child partly buffers the negative effect of high mobile device screen time on language comprehension difficulties but not on expressive language skills difficulties. No modifying effect of parental education and time spent by the child on TV/PC was found. CONCLUSIONS: Mobile device screen time of one hour or more per day is associated with poorer language development among toddlers. Reading frequently to the child may have a buffering effect on language comprehension difficulties but not on expressive language skills difficulties.

Automatic inference of ICD-10 codes from German ophthalmologic physicians' letters using natural language processing.

Physicians' letters are the optimal source of diagnoses for registries. However, most registries demand for diagnosis codes such as ICD-10. We herein describe an algorithm that infers ICD-10 codes from German ophthalmologic physicians' letters. We assess the method in three German eye hospitals. Our algorithm is based on the nearest-neighbor method as well as on a large thesaurus for ICD-10 codes. This thesaurus was embedded into a Word2Vec space created from anonymized physicians' reports of the first hospital. For evaluation, each of the three hospitals sent all diagnoses taken from 100 letters. The inferred ICD-10 codes were evaluated for correctness by the senders. A total of 3332 natural language terms had been sent in (812 hospital one, 1473 hospital two, 1047 hospital three). A total of 526 non-diagnoses were excluded upfront. 2806 ICD-10 codes were inferred (771 hospital one, 1226 hospital two, 809 hospital three). In the first hospital, 98% were fully correct and 99% correct at the level of the superordinate disease concept. The percentages in hospital two were 69% and 86%. The respective numbers for hospital three were 69% and 91%. Our simple method is capable of inferring ICD-10 codes for German natural language diagnoses, especially when the embedding space has been built with physicians' letters from the same hospital. The method may yield sufficient accuracy for many tasks in the multi-centric setting and can easily be adapted to other languages/specialities.

Distributions of recorded pain in mental health records: a natural language processing based study.

OBJECTIVE: The objective of this study is to determine demographic and diagnostic distributions of physical pain recorded in clinical notes of a mental health electronic health records database by using natural language processing and examine the overlap in recorded physical pain between primary and secondary care. DESIGN, SETTING AND PARTICIPANTS: The data were extracted from an anonymised version of the electronic health records of a large secondary mental healthcare provider serving a catchment of 1.3 million residents in south London. These included patients under active referral, aged 18+ at the index date of 1 July 2018 and having at least one clinical document (≥30 characters) between 1 July 2017 and 1 July 2019. This cohort was compared with linked primary care records from one of the four local government areas. OUTCOME: The primary outcome of interest was the presence of recorded physical pain within the clinical notes of the patients, not including psychological or metaphorical pain. RESULTS: A total of 27 211 patients were retrieved. Of these, 52% (14,202) had narrative text containing relevant mentions of physical pain. Older patients (OR 1.17, 95% CI 1.15 to 1.19), females (OR 1.42, 95% CI 1.35 to 1.49), Asians (OR 1.30, 95% CI 1.16 to 1.45) or black (OR 1.49, 95% CI 1.40 to 1.59) ethnicities, living in deprived neighbourhoods (OR 1.64, 95% CI 1.55 to 1.73) showed higher odds of recorded pain. Patients with severe mental illnesses were found to be less likely to report pain (OR 0.43, 95% CI 0.41 to 0.46, p<0.001). 17% of the cohort from secondary care also had records from primary care. CONCLUSION: The findings of this study show sociodemographic and diagnostic differences in recorded pain. Specifically, lower documentation across certain groups indicates the need for better screening protocols and training on recognising varied pain presentations. Additionally, targeting improved detection of pain for minority and disadvantaged groups by care providers can promote health equity.

Enhancing Early Detection of Cognitive Decline in the Elderly through Ensemble of NLP Techniques: A Comparative Study Utilizing Large Language Models in Clinical Notes.

Background: Early detection of cognitive decline in elderly individuals facilitates clinical trial enrollment and timely medical interventions. This study aims to apply, evaluate, and compare advanced natural language processing techniques for identifying signs of cognitive decline in clinical notes. Methods: This study, conducted at Mass General Brigham (MGB), Boston, MA, included clinical notes from the 4 years prior to initial mild cognitive impairment (MCI) diagnosis in 2019 for patients ≥ 50 years. Note sections regarding cognitive decline were labeled manually. A random sample of 4,949 note sections filtered with cognitive functions-related keywords were used for traditional AI model development, and 200 random subset were used for LLM and prompt development; another random sample of 1996 note sections without keyword filtering were used for testing. Prompt templates for large language models (LLM), Llama 2 on Amazon Web Service and GPT-4 on Microsoft Azure, were developed with multiple prompting approaches to select the optimal LLM-based method. Baseline comparisons were made with XGBoost and a hierarchical attention-based deep neural network model. An ensemble of the three models was then constructed using majority vote. Results: GPT-4 demonstrated superior accuracy and efficiency to Llama 2. The ensemble model outperformed individual models, achieving a precision of 90.3%, recall of 94.2%, and F1-score of 92.2%. Notably, the ensemble model demonstrated a marked improvement in precision (from a 70%-79% range to above 90%) compared to the best performing single model. Error analysis revealed 63 samples were wrongly predicted by at least one model; however, only 2 cases (3.2%) were mutual errors across all models, indicating diverse error profiles among them. Conclusion: Our findings indicate that LLMs and traditional models exhibit diverse error profiles. The ensemble of LLMs and locally trained machine learning models on EHR data was found to be complementary, enhancing performance and improving diagnostic accuracy.

A Comparison Between GPT-3.5, GPT-4, and GPT-4V: Can the Large Language Model (ChatGPT) Pass the Japanese Board of Orthopaedic Surgery Examination?

Introduction Recently, large-scale language models, such as ChatGPT (OpenAI, San Francisco, CA), have evolved. These models are designed to think and act like humans and possess a broad range of specialized knowledge. GPT-3.5 was reported to be at a level of passing the United States Medical Licensing Examination. Its capabilities continue to evolve, and in October 2023, GPT-4V became available as a model capable of image recognition. Therefore, it is important to know the current performance of these models because they will be soon incorporated into medical practice. We aimed to evaluate the performance of ChatGPT in the field of orthopedic surgery. Methods We used three years' worth of Japanese Board of Orthopaedic Surgery Examinations (JBOSE) conducted in 2021, 2022, and 2023. Questions and their multiple-choice answers were used in their original Japanese form, as was the official examination rubric. We inputted these questions into three versions of ChatGPT: GPT-3.5, GPT-4, and GPT-4V. For image-based questions, we inputted only textual statements for GPT-3.5 and GPT-4, and both image and textual statements for GPT-4V. As the minimum scoring rate acquired to pass is not officially disclosed, it was calculated using publicly available data. Results The estimated minimum scoring rate acquired to pass was calculated as 50.1% (43.7-53.8%). For GPT-4, even when answering all questions, including the image-based ones, the percentage of correct answers was 59% (55-61%) and GPT-4 was able to achieve the passing line. When excluding image-based questions, the score reached 67% (63-73%). For GPT-3.5, the percentage was limited to 30% (28-32%), and this version could not pass the examination. There was a significant difference in the performance between GPT-4 and GPT-3.5 (p < 0.001). For image-based questions, the percentage of correct answers was 25% in GPT-3.5, 38% in GPT-4, and 38% in GPT-4V. There was no significant difference in the performance for image-based questions between GPT-4 and GPT-4V. Conclusions ChatGPT had enough performance to pass the orthopedic specialist examination. After adding further training data such as images, ChatGPT is expected to be applied to the orthopedics field.

Evaluation of Large Language Model Performance and Reliability for Citations and References in Scholarly Writing: Cross-Disciplinary Study.

BACKGROUND: Large language models (LLMs) have gained prominence since the release of ChatGPT in late 2022. OBJECTIVE: The aim of this study was to assess the accuracy of citations and references generated by ChatGPT (GPT-3.5) in two distinct academic domains: the natural sciences and humanities. METHODS: Two researchers independently prompted ChatGPT to write an introduction section for a manuscript and include citations; they then evaluated the accuracy of the citations and Digital Object Identifiers (DOIs). Results were compared between the two disciplines. RESULTS: Ten topics were included, including 5 in the natural sciences and 5 in the humanities. A total of 102 citations were generated, with 55 in the natural sciences and 47 in the humanities. Among these, 40 citations (72.7%) in the natural sciences and 36 citations (76.6%) in the humanities were confirmed to exist (P=.42). There were significant disparities found in DOI presence in the natural sciences (39/55, 70.9%) and the humanities (18/47, 38.3%), along with significant differences in accuracy between the two disciplines (18/55, 32.7% vs 4/47, 8.5%). DOI hallucination was more prevalent in the humanities (42/55, 89.4%). The Levenshtein distance was significantly higher in the humanities than in the natural sciences, reflecting the lower DOI accuracy. CONCLUSIONS: ChatGPT's performance in generating citations and references varies across disciplines. Differences in DOI standards and disciplinary nuances contribute to performance variations. Researchers should consider the strengths and limitations of artificial intelligence writing tools with respect to citation accuracy. The use of domain-specific models may enhance accuracy.